Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Set test framework and test framework version tags atomically #8252

Open
wants to merge 5 commits into
base: master
Choose a base branch
from
Open

Set test framework and test framework version tags atomically #8252

wants to merge 5 commits into from
+24 −4

Conversation

nikita-tkachenko-datadog
Copy link
Contributor

What Does This Do

Updates the logic that sets test.framework and test.framework_version tags to do it atomically.

Motivation

When test framework name and version are propagated from test to test suite span, some merging is done.
It is possible for multiple tests from the same suite to be executed in parallel, which is why the two tags need to be set atomically.

Contributor Checklist

Jira ticket: [PROJ-IDENT]

@nikita-tkachenko-datadog nikita-tkachenko-datadog added type: bug comp: ci visibility Continuous Integration Visibility labels Jan 20, 2025
@pr-commenter
Copy link

pr-commenter bot commented Jan 20, 2025 •
edited
Loading

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/test-framework-tags-sync
git_commit_date 1737638844 1737643803
git_commit_sha ac29d6c d81799f
release_version 1.46.0-SNAPSHOT~ac29d6cfa4 1.46.0-SNAPSHOT~d81799fda4
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1737646152 1737646152
ci_job_id 777822516 777822516
ci_pipeline_id 53803885 53803885
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-fgow3nxt-project-304-concurrent-3-utfztxdh 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-fgow3nxt-project-304-concurrent-3-utfztxdh 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 57 metrics, 6 unstable metrics.

Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.46.0-SNAPSHOT~d81799fda4, baseline=1.46.0-SNAPSHOT~ac29d6cfa4

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.059 s) : 0, 1058904
Total [baseline] (10.538 s) : 0, 10537790
Agent [candidate] (1.063 s) : 0, 1063157
Total [candidate] (10.468 s) : 0, 10468315
section appsec
Agent [baseline] (1.189 s) : 0, 1188544
Total [baseline] (10.736 s) : 0, 10735505
Agent [candidate] (1.192 s) : 0, 1191878
Total [candidate] (10.738 s) : 0, 10737998
section iast
Agent [baseline] (1.184 s) : 0, 1184373
Total [baseline] (10.954 s) : 0, 10953929
Agent [candidate] (1.195 s) : 0, 1195409
Total [candidate] (10.927 s) : 0, 10926828
section profiling
Agent [baseline] (1.253 s) : 0, 1253381
Total [baseline] (10.73 s) : 0, 10730296
Agent [candidate] (1.264 s) : 0, 1264446
Total [candidate] (10.792 s) : 0, 10791744
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.059 s -
Agent appsec 1.189 s 129.64 ms (12.2%)
Agent iast 1.184 s 125.469 ms (11.8%)
Agent profiling 1.253 s 194.477 ms (18.4%)
Total tracing 10.538 s -
Total appsec 10.736 s 197.715 ms (1.9%)
Total iast 10.954 s 416.139 ms (3.9%)
Total profiling 10.73 s 192.506 ms (1.8%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.063 s -
Agent appsec 1.192 s 128.721 ms (12.1%)
Agent iast 1.195 s 132.251 ms (12.4%)
Agent profiling 1.264 s 201.289 ms (18.9%)
Total tracing 10.468 s -
Total appsec 10.738 s 269.684 ms (2.6%)
Total iast 10.927 s 458.514 ms (4.4%)
Total profiling 10.792 s 323.429 ms (3.1%) 
gantt
    title petclinic - break down per module: candidate=1.46.0-SNAPSHOT~d81799fda4, baseline=1.46.0-SNAPSHOT~ac29d6cfa4

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (716.228 ms) : 0, 716228
BytebuddyAgent [candidate] (720.308 ms) : 0, 720308
GlobalTracer [baseline] (257.276 ms) : 0, 257276
GlobalTracer [candidate] (256.782 ms) : 0, 256782
AppSec [baseline] (55.965 ms) : 0, 55965
AppSec [candidate] (55.842 ms) : 0, 55842
Remote Config [baseline] (750.442 µs) : 0, 750
Remote Config [candidate] (749.315 µs) : 0, 749
Telemetry [baseline] (13.702 ms) : 0, 13702
Telemetry [candidate] (14.329 ms) : 0, 14329
section appsec
BytebuddyAgent [baseline] (731.851 ms) : 0, 731851
BytebuddyAgent [candidate] (733.215 ms) : 0, 733215
GlobalTracer [baseline] (252.162 ms) : 0, 252162
GlobalTracer [candidate] (253.637 ms) : 0, 253637
AppSec [baseline] (171.101 ms) : 0, 171101
AppSec [candidate] (171.174 ms) : 0, 171174
Remote Config [baseline] (655.593 µs) : 0, 656
Remote Config [candidate] (671.25 µs) : 0, 671
Telemetry [baseline] (8.115 ms) : 0, 8115
Telemetry [candidate] (8.485 ms) : 0, 8485
IAST [baseline] (19.289 ms) : 0, 19289
IAST [candidate] (19.265 ms) : 0, 19265
section iast
BytebuddyAgent [baseline] (833.173 ms) : 0, 833173
BytebuddyAgent [candidate] (841.402 ms) : 0, 841402
GlobalTracer [baseline] (246.757 ms) : 0, 246757
GlobalTracer [candidate] (248.956 ms) : 0, 248956
AppSec [baseline] (58.355 ms) : 0, 58355
AppSec [candidate] (58.569 ms) : 0, 58569
Remote Config [baseline] (668.459 µs) : 0, 668
Remote Config [candidate] (666.606 µs) : 0, 667
Telemetry [baseline] (8.776 ms) : 0, 8776
Telemetry [candidate] (8.834 ms) : 0, 8834
IAST [baseline] (21.616 ms) : 0, 21616
IAST [candidate] (21.779 ms) : 0, 21779
section profiling
ProfilingAgent [baseline] (95.182 ms) : 0, 95182
ProfilingAgent [candidate] (96.828 ms) : 0, 96828
BytebuddyAgent [baseline] (703.914 ms) : 0, 703914
BytebuddyAgent [candidate] (707.921 ms) : 0, 707921
GlobalTracer [baseline] (347.992 ms) : 0, 347992
GlobalTracer [candidate] (353.309 ms) : 0, 353309
AppSec [baseline] (54.897 ms) : 0, 54897
AppSec [candidate] (54.454 ms) : 0, 54454
Remote Config [baseline] (725.555 µs) : 0, 726
Remote Config [candidate] (734.636 µs) : 0, 735
Telemetry [baseline] (8.743 ms) : 0, 8743
Telemetry [candidate] (8.93 ms) : 0, 8930
Profiling [baseline] (95.206 ms) : 0, 95206
Profiling [candidate] (96.852 ms) : 0, 96852
Loading
Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.46.0-SNAPSHOT~d81799fda4, baseline=1.46.0-SNAPSHOT~ac29d6cfa4

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.065 s) : 0, 1064711
Total [baseline] (8.651 s) : 0, 8651223
Agent [candidate] (1.077 s) : 0, 1076826
Total [candidate] (8.722 s) : 0, 8721900
section iast
Agent [baseline] (1.184 s) : 0, 1183707
Total [baseline] (9.231 s) : 0, 9230504
Agent [candidate] (1.186 s) : 0, 1185864
Total [candidate] (9.203 s) : 0, 9203017
section iast_HARDCODED_SECRET_DISABLED
Agent [baseline] (1.183 s) : 0, 1182865
Total [baseline] (9.184 s) : 0, 9184212
Agent [candidate] (1.185 s) : 0, 1184908
Total [candidate] (9.169 s) : 0, 9168917
section iast_TELEMETRY_OFF
Agent [baseline] (1.178 s) : 0, 1177883
Total [baseline] (9.167 s) : 0, 9167194
Agent [candidate] (1.179 s) : 0, 1178971
Total [candidate] (9.188 s) : 0, 9188089
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.065 s -
Agent iast 1.184 s 118.996 ms (11.2%)
Agent iast_HARDCODED_SECRET_DISABLED 1.183 s 118.154 ms (11.1%)
Agent iast_TELEMETRY_OFF 1.178 s 113.172 ms (10.6%)
Total tracing 8.651 s -
Total iast 9.231 s 579.281 ms (6.7%)
Total iast_HARDCODED_SECRET_DISABLED 9.184 s 532.989 ms (6.2%)
Total iast_TELEMETRY_OFF 9.167 s 515.971 ms (6.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.077 s -
Agent iast 1.186 s 109.038 ms (10.1%)
Agent iast_HARDCODED_SECRET_DISABLED 1.185 s 108.082 ms (10.0%)
Agent iast_TELEMETRY_OFF 1.179 s 102.145 ms (9.5%)
Total tracing 8.722 s -
Total iast 9.203 s 481.117 ms (5.5%)
Total iast_HARDCODED_SECRET_DISABLED 9.169 s 447.018 ms (5.1%)
Total iast_TELEMETRY_OFF 9.188 s 466.189 ms (5.3%) 
gantt
    title insecure-bank - break down per module: candidate=1.46.0-SNAPSHOT~d81799fda4, baseline=1.46.0-SNAPSHOT~ac29d6cfa4

    dateFormat X
    axisFormat %s
section tracing
BytebuddyAgent [baseline] (721.115 ms) : 0, 721115
BytebuddyAgent [candidate] (729.192 ms) : 0, 729192
GlobalTracer [baseline] (257.037 ms) : 0, 257037
GlobalTracer [candidate] (260.273 ms) : 0, 260273
AppSec [baseline] (55.744 ms) : 0, 55744
AppSec [candidate] (56.543 ms) : 0, 56543
Remote Config [baseline] (758.831 µs) : 0, 759
Remote Config [candidate] (785.697 µs) : 0, 786
Telemetry [baseline] (14.944 ms) : 0, 14944
Telemetry [candidate] (14.677 ms) : 0, 14677
section iast
BytebuddyAgent [baseline] (832.533 ms) : 0, 832533
BytebuddyAgent [candidate] (833.945 ms) : 0, 833945
GlobalTracer [baseline] (246.392 ms) : 0, 246392
GlobalTracer [candidate] (247.709 ms) : 0, 247709
AppSec [baseline] (58.37 ms) : 0, 58370
AppSec [candidate] (58.16 ms) : 0, 58160
IAST [baseline] (21.871 ms) : 0, 21871
IAST [candidate] (21.608 ms) : 0, 21608
Remote Config [baseline] (686.188 µs) : 0, 686
Remote Config [candidate] (679.173 µs) : 0, 679
Telemetry [baseline] (8.784 ms) : 0, 8784
Telemetry [candidate] (8.721 ms) : 0, 8721
section iast_HARDCODED_SECRET_DISABLED
BytebuddyAgent [baseline] (832.05 ms) : 0, 832050
BytebuddyAgent [candidate] (833.279 ms) : 0, 833279
GlobalTracer [baseline] (246.15 ms) : 0, 246150
GlobalTracer [candidate] (247.207 ms) : 0, 247207
AppSec [baseline] (58.305 ms) : 0, 58305
AppSec [candidate] (58.229 ms) : 0, 58229
IAST [baseline] (21.746 ms) : 0, 21746
IAST [candidate] (21.663 ms) : 0, 21663
Remote Config [baseline] (678.477 µs) : 0, 678
Remote Config [candidate] (667.742 µs) : 0, 668
Telemetry [baseline] (8.849 ms) : 0, 8849
Telemetry [candidate] (8.781 ms) : 0, 8781
section iast_TELEMETRY_OFF
BytebuddyAgent [baseline] (829.007 ms) : 0, 829007
BytebuddyAgent [candidate] (828.736 ms) : 0, 828736
GlobalTracer [baseline] (245.926 ms) : 0, 245926
GlobalTracer [candidate] (246.619 ms) : 0, 246619
AppSec [baseline] (57.979 ms) : 0, 57979
AppSec [candidate] (58.147 ms) : 0, 58147
IAST [baseline] (20.752 ms) : 0, 20752
IAST [candidate] (21.118 ms) : 0, 21118
Remote Config [baseline] (664.692 µs) : 0, 665
Remote Config [candidate] (671.303 µs) : 0, 671
Telemetry [baseline] (8.593 ms) : 0, 8593
Telemetry [candidate] (8.708 ms) : 0, 8708
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
end_time 2025-01-23T14:59:14 2025-01-23T15:06:17
git_branch master nikita-tkachenko/test-framework-tags-sync
git_commit_date 1737638844 1737643803
git_commit_sha ac29d6c d81799f
release_version 1.46.0-SNAPSHOT~ac29d6cfa4 1.46.0-SNAPSHOT~d81799fda4
start_time 2025-01-23T14:59:00 2025-01-23T15:06:03
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1737645135 1737645135
ci_job_id 777822517 777822517
ci_pipeline_id 53803885 53803885
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-fgow3nxt-project-304-concurrent-4-cvjn53c9 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-fgow3nxt-project-304-concurrent-4-cvjn53c9 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant iast iast

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 17 unstable metrics.

Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~d81799fda4, baseline=1.46.0-SNAPSHOT~ac29d6cfa4
    dateFormat X
    axisFormat %s
section baseline
no_agent (377.21 µs) : 358, 397
.   : milestone, 377,
iast (509.162 µs) : 488, 531
.   : milestone, 509,
iast_FULL (742.578 µs) : 720, 765
.   : milestone, 743,
iast_GLOBAL (554.359 µs) : 533, 576
.   : milestone, 554,
iast_HARDCODED_SECRET_DISABLED (507.432 µs) : 486, 529
.   : milestone, 507,
iast_INACTIVE (464.727 µs) : 443, 487
.   : milestone, 465,
iast_TELEMETRY_OFF (501.461 µs) : 479, 524
.   : milestone, 501,
tracing (453.136 µs) : 432, 475
.   : milestone, 453,
section candidate
no_agent (382.256 µs) : 363, 402
.   : milestone, 382,
iast (510.106 µs) : 488, 532
.   : milestone, 510,
iast_FULL (748.058 µs) : 726, 770
.   : milestone, 748,
iast_GLOBAL (556.531 µs) : 535, 578
.   : milestone, 557,
iast_HARDCODED_SECRET_DISABLED (504.423 µs) : 483, 526
.   : milestone, 504,
iast_INACTIVE (459.336 µs) : 438, 481
.   : milestone, 459,
iast_TELEMETRY_OFF (495.71 µs) : 474, 517
.   : milestone, 496,
tracing (454.41 µs) : 434, 475
.   : milestone, 454,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 377.21 µs [357.541 µs, 396.879 µs] -
iast 509.162 µs [487.716 µs, 530.607 µs] 131.952 µs (35.0%)
iast_FULL 742.578 µs [720.187 µs, 764.968 µs] 365.368 µs (96.9%)
iast_GLOBAL 554.359 µs [532.829 µs, 575.89 µs] 177.149 µs (47.0%)
iast_HARDCODED_SECRET_DISABLED 507.432 µs [485.669 µs, 529.195 µs] 130.223 µs (34.5%)
iast_INACTIVE 464.727 µs [442.664 µs, 486.791 µs] 87.517 µs (23.2%)
iast_TELEMETRY_OFF 501.461 µs [479.189 µs, 523.733 µs] 124.251 µs (32.9%)
tracing 453.136 µs [431.742 µs, 474.529 µs] 75.926 µs (20.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 382.256 µs [362.619 µs, 401.893 µs] -
iast 510.106 µs [488.182 µs, 532.031 µs] 127.85 µs (33.4%)
iast_FULL 748.058 µs [726.105 µs, 770.011 µs] 365.802 µs (95.7%)
iast_GLOBAL 556.531 µs [534.592 µs, 578.47 µs] 174.275 µs (45.6%)
iast_HARDCODED_SECRET_DISABLED 504.423 µs [482.661 µs, 526.185 µs] 122.167 µs (32.0%)
iast_INACTIVE 459.336 µs [437.645 µs, 481.027 µs] 77.08 µs (20.2%)
iast_TELEMETRY_OFF 495.71 µs [474.104 µs, 517.316 µs] 113.454 µs (29.7%)
tracing 454.41 µs [433.541 µs, 475.279 µs] 72.154 µs (18.9%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.46.0-SNAPSHOT~d81799fda4, baseline=1.46.0-SNAPSHOT~ac29d6cfa4
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.352 ms) : 1332, 1372
.   : milestone, 1352,
appsec (1.729 ms) : 1705, 1753
.   : milestone, 1729,
appsec_no_iast (1.748 ms) : 1722, 1773
.   : milestone, 1748,
iast (1.514 ms) : 1489, 1538
.   : milestone, 1514,
profiling (1.521 ms) : 1497, 1546
.   : milestone, 1521,
tracing (1.475 ms) : 1449, 1501
.   : milestone, 1475,
section candidate
no_agent (1.37 ms) : 1350, 1389
.   : milestone, 1370,
appsec (1.739 ms) : 1715, 1763
.   : milestone, 1739,
appsec_no_iast (1.753 ms) : 1728, 1777
.   : milestone, 1753,
iast (1.515 ms) : 1490, 1540
.   : milestone, 1515,
profiling (1.521 ms) : 1497, 1544
.   : milestone, 1521,
tracing (1.483 ms) : 1458, 1507
.   : milestone, 1483,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.352 ms [1.332 ms, 1.372 ms] -
appsec 1.729 ms [1.705 ms, 1.753 ms] 377.156 µs (27.9%)
appsec_no_iast 1.748 ms [1.722 ms, 1.773 ms] 395.859 µs (29.3%)
iast 1.514 ms [1.489 ms, 1.538 ms] 161.789 µs (12.0%)
profiling 1.521 ms [1.497 ms, 1.546 ms] 169.477 µs (12.5%)
tracing 1.475 ms [1.449 ms, 1.501 ms] 122.991 µs (9.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.37 ms [1.35 ms, 1.389 ms] -
appsec 1.739 ms [1.715 ms, 1.763 ms] 369.385 µs (27.0%)
appsec_no_iast 1.753 ms [1.728 ms, 1.777 ms] 383.021 µs (28.0%)
iast 1.515 ms [1.49 ms, 1.54 ms] 145.715 µs (10.6%)
profiling 1.521 ms [1.497 ms, 1.544 ms] 150.985 µs (11.0%)
tracing 1.483 ms [1.458 ms, 1.507 ms] 112.858 µs (8.2%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master nikita-tkachenko/test-framework-tags-sync
git_commit_date 1737638844 1737643803
git_commit_sha ac29d6c d81799f
release_version 1.46.0-SNAPSHOT~ac29d6cfa4 1.46.0-SNAPSHOT~d81799fda4
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1737645829 1737645829
ci_job_id 777822518 777822518
ci_pipeline_id 53803885 53803885
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-h9qrujh-project-304-concurrent-1-isaswn1c 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux Linux runner-h9qrujh-project-304-concurrent-1-isaswn1c 6.8.0-1018-aws #19~22.04.1-Ubuntu SMP Wed Oct 9 16:48:22 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
variant appsec appsec

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 12 metrics, 0 unstable metrics.

Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~d81799fda4, baseline=1.46.0-SNAPSHOT~ac29d6cfa4
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.683 s) : 14683000, 14683000
.   : milestone, 14683000,
appsec (14.901 s) : 14901000, 14901000
.   : milestone, 14901000,
iast (18.528 s) : 18528000, 18528000
.   : milestone, 18528000,
iast_GLOBAL (18.244 s) : 18244000, 18244000
.   : milestone, 18244000,
profiling (15.1 s) : 15100000, 15100000
.   : milestone, 15100000,
tracing (14.852 s) : 14852000, 14852000
.   : milestone, 14852000,
section candidate
no_agent (15.378 s) : 15378000, 15378000
.   : milestone, 15378000,
appsec (15.113 s) : 15113000, 15113000
.   : milestone, 15113000,
iast (18.779 s) : 18779000, 18779000
.   : milestone, 18779000,
iast_GLOBAL (18.025 s) : 18025000, 18025000
.   : milestone, 18025000,
profiling (15.1 s) : 15100000, 15100000
.   : milestone, 15100000,
tracing (15.036 s) : 15036000, 15036000
.   : milestone, 15036000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.683 s [14.683 s, 14.683 s] -
appsec 14.901 s [14.901 s, 14.901 s] 218.0 ms (1.5%)
iast 18.528 s [18.528 s, 18.528 s] 3.845 s (26.2%)
iast_GLOBAL 18.244 s [18.244 s, 18.244 s] 3.561 s (24.3%)
profiling 15.1 s [15.1 s, 15.1 s] 417.0 ms (2.8%)
tracing 14.852 s [14.852 s, 14.852 s] 169.0 ms (1.2%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.378 s [15.378 s, 15.378 s] -
appsec 15.113 s [15.113 s, 15.113 s] -265.0 ms (-1.7%)
iast 18.779 s [18.779 s, 18.779 s] 3.401 s (22.1%)
iast_GLOBAL 18.025 s [18.025 s, 18.025 s] 2.647 s (17.2%)
profiling 15.1 s [15.1 s, 15.1 s] -278.0 ms (-1.8%)
tracing 15.036 s [15.036 s, 15.036 s] -342.0 ms (-2.2%)
Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.46.0-SNAPSHOT~d81799fda4, baseline=1.46.0-SNAPSHOT~ac29d6cfa4
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.47 ms) : 1459, 1482
.   : milestone, 1470,
appsec (2.361 ms) : 2318, 2404
.   : milestone, 2361,
iast (2.108 ms) : 2053, 2163
.   : milestone, 2108,
iast_GLOBAL (2.151 ms) : 2096, 2205
.   : milestone, 2151,
profiling (1.976 ms) : 1931, 2021
.   : milestone, 1976,
tracing (1.947 ms) : 1905, 1989
.   : milestone, 1947,
section candidate
no_agent (1.474 ms) : 1462, 1485
.   : milestone, 1474,
appsec (2.362 ms) : 2319, 2405
.   : milestone, 2362,
iast (2.101 ms) : 2047, 2155
.   : milestone, 2101,
iast_GLOBAL (2.151 ms) : 2096, 2205
.   : milestone, 2151,
profiling (1.966 ms) : 1923, 2009
.   : milestone, 1966,
tracing (1.943 ms) : 1901, 1985
.   : milestone, 1943,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.47 ms [1.459 ms, 1.482 ms] -
appsec 2.361 ms [2.318 ms, 2.404 ms] 891.198 µs (60.6%)
iast 2.108 ms [2.053 ms, 2.163 ms] 637.935 µs (43.4%)
iast_GLOBAL 2.151 ms [2.096 ms, 2.205 ms] 680.519 µs (46.3%)
profiling 1.976 ms [1.931 ms, 2.021 ms] 506.034 µs (34.4%)
tracing 1.947 ms [1.905 ms, 1.989 ms] 476.652 µs (32.4%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.474 ms [1.462 ms, 1.485 ms] -
appsec 2.362 ms [2.319 ms, 2.405 ms] 888.0 µs (60.3%)
iast 2.101 ms [2.047 ms, 2.155 ms] 627.433 µs (42.6%)
iast_GLOBAL 2.151 ms [2.096 ms, 2.205 ms] 676.926 µs (45.9%)
profiling 1.966 ms [1.923 ms, 2.009 ms] 492.28 µs (33.4%)
tracing 1.943 ms [1.901 ms, 1.985 ms] 469.242 µs (31.8%)

@nikita-tkachenko-datadog nikita-tkachenko-datadog marked this pull request as ready for review January 20, 2025 13:38
@nikita-tkachenko-datadog nikita-tkachenko-datadog force-pushed the nikita-tkachenko/test-framework-tags-sync branch from 20a5542 to a83cee4 Compare January 22, 2025 16:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@juan-fernandez juan-fernandez juan-fernandez approved these changes

@anmarchenko anmarchenko anmarchenko approved these changes

@ygree ygree Awaiting requested review from ygree ygree is a code owner automatically assigned from DataDog/apm-java

Assignees
No one assigned
Labels
comp: ci visibility Continuous Integration Visibility type: bug
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@nikita-tkachenko-datadog @anmarchenko @juan-fernandez